Unique patterns extracted from involuntary eye motions to identify individuals

ABSTRACT

A method for user authentication is disclosed including capturing involuntary eye movement of an eyeball of a user; generating a unique pattern to identify the user in response to the involuntary eye movement; storing the unique pattern into a secured non-volatile memory device; and authenticating the user with an electronic device in response to the stored unique pattern.

CROSS REFERENCE

This United States (U.S.) patent application claims the benefit of U.S.Non-provisional patent application Ser. No. 15/806,094 titled UNIQUEPATTERNS EXTRACTED FROM INVOLUNTARY EYE MOTIONS TO IDENTIFY INDIVIDUALSfiled on Nov. 7, 2017 by inventors Martin Zizi et al. U.S. patentapplication Ser. No. 15/806,094 claims the benefit of U.S. ProvisionalPatent Application No. 62/419,458 titled UNIQUE PATTERNS EXTRACTED FROMINVOLUNTARY EYE MOTIONS TO IDENTIFY INDIVIDUALS filed on Nov. 8, 2016 byinventors Martin Zizi et al.

FIELD

The embodiments of the invention relate generally to user identificationand authentication.

BACKGROUND

Referring now to FIG. 1 , a cross sectional view of a human eyeball 100is shown within a skull 102. The human eyeball 100 is an imperfect globethat can be moved within the skull 102 by a plurality of muscles. Theact or process of change in position of the human eyeball 100 within theskull 102 is referred to as eye movement or eye motion.

The eyeball 100 includes a retina 110, a pupil 112, an iris 114, a fovea116, and a lens 118 that interact to capture color images for processingby the brain. A cornea 122 of the eyeball supports the pupil 112, iris114, and lens 118 over the retina 110. The pupil 112 alters its diameterto adjust the amount of light received by the retina 110.

The retina 110 of the eyeball includes two types of photoreceptors, rodsand cones. There are around 120 million cones and 6 to 7 million rods inthe retina 110. Cones are concentrated in a rod free area of the retinareferred to as the fovea centralis or macula that provides for maximumacuity and color sensitivity. The cones are smaller and more closelypacked than elsewhere on the retina 110.

The optic nerve 126 is a cable of nerve fibers coupled to the eyeballthat carries electrical signals from the rods and cones in the retina tothe brain. The point where the optic nerve departs the eyeball throughthe retina is devoid of rods and cones. Thus, the optic nerve forms a“blind spot” in the retina.

FIGS. 2A-2C illustrate the plurality of muscles 202-202 coupled to theeyeball 100 to cause eye movement within the skull. In FIG. 2A, a leftlateral rectus muscle 201L and a right lateral rectus muscle 201R pivotand move the eyeball 100 left and right horizontally as shown by thearrowhead 211. In FIG. 2B, a superior rectus muscle 201T on top and aninferior rectus muscle 201B on bottom pivot and move the eyeball 100 upand dawn vertically as shown by the arrowhead 212. In FIG. 2C, asuperior oblique muscle 202S and an inferior oblique muscle 202I rolland move the eyeball 100 as shown by the curved arrowhead 213. Thesemuscles can cause voluntary eye movement under the will of a human beingand involuntary eye movement that the human being does not even know hasoccurred. Other muscles around the eyeball 100 may also contribute tovoluntary and involuntary eye movement. The muscles are under control ofthe nervous system in a body including the brain.

Involuntary eye movement, in contrast to voluntary eye movement, isoften considered to be a pathologic condition when observed clinicallyby an eye doctor. However, there are normal, physiologic,miniature-involuntary eye movements that occur and are more oftenobserved during eye fixation on a target. These miniature-involuntaryeye movements are a normal physiological function of the body to preventfatigue of the rods and cones at the focal point on the retinal surfacewithin the eyeball. Involuntary eye motions have typically beenconsidered to be disturbances or analyzed for study purposes.

The retina 110 of the human eyeball 100 may scanned for variouspurposes. Retinal scanners that capture a two dimensional map of ananatomy of a retina of an eye are known. Retinal scanners were notintended to measure involuntary eye motion.

Various eye-tracking systems have been used to detect voluntary eyemovement for a variety of purposes. For example, virtual realityheadsets for gaming may track voluntary eye motion in game play of avideo game. As another example, heads-up displays for military systemsmay track voluntary eye motion for some military purposes. However, eyetracking systems were intended to assess the foveal visual field and thefocal attention of the subject and not measure involuntary eye motion,when the involuntary eye motions were considered to be eitherdisturbances or for study purposes only.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 is a cross section of an eyeball in the eye socket of a humanskull.

FIGS. 2A-2C are diagrams illustrating the various muscles coupled to theeyeball that cause various eye movements of the eyeball.

FIG. 3 is a diagram of the fixation process for capturing involuntaryeye movements of the eyeball.

FIGS. 4A-4B are diagrams of magnified portions of the retina with agraph of involuntary eye movement charted over the zones of the cones inresponse to a user fixating on a target for a period of time.

FIGS. 5A-5C are diagrams illustrating various combinations of theinvoluntary eye movements that can be used for user identification andauthentication.

FIGS. 6A-6C are diagrams of the viewable features in video images of theeyeball that may be used to detect involuntary eye movements.

FIG. 7 is a block diagram of electrooculography system to directlygenerate signals of involuntary eye movements of a user.

FIG. 8A is a diagram of an electronic device with a video camera thatmay be used to capture images of eye movement of a user.

FIG. 8B is a block diagram of the electronic device with the videocamera shown in FIG. 8A that may be used to capture images of eyemovement of the user.

FIGS. 9A-9B are diagrams of electronic glasses with a video camera thatmay be used to capture images of eye movement of a user.

FIG. 9C is a magnified view of a target that may be used with theelectronic glasses of FIGS. 9A-9B.

FIGS. 10A-10B are diagrams of virtual reality goggles with a videocamera that may be used to capture images of eye movement of a user.

FIG. 11A is a diagram of an eyeball with a contact lens with an emitterdevice that may be used to aid in the capture of data regarding eyemovement of a user.

FIGS. 11B-11D are diagrams that illustrate contact lenses with variousemitter devices that may be used to aid in the capture of data regardingeye movement of a user.

FIG. 11E is a functional block diagram of an active emitter device.

FIG. 11F is a side view of the sensing device near the contact lensmounted to the eyeball that may be used to capture data regarding eyemovement of a user.

FIGS. 12A-12B illustrate an eye motion capture device affixed to abuilding structure to control access thereto.

FIGS. 13A-13B illustrate a stand alone eye scanner to authenticate auser to a system.

DETAILED DESCRIPTION

In the following detailed description of the embodiments of theinvention, numerous specific details are set forth in order to provide athorough understanding. However, it will be obvious to one skilled inthe art that the embodiments may be practiced without these specificdetails. In other instances well known methods, procedures, components,and circuits have not been described in detail so as not tounnecessarily obscure aspects of the embodiments.

Retinal sensitivity to photons is defined by genetic factors amongothers, by eye motions (eye movement), by eyeball muscles, and theintegration of the whole system by specific brain wirings. Miniatureinvoluntary eye movements of the eyeball 100 are a normal physiologicalfunction of the human body. Any neurologically mediated process willproduce features that are unique (“proprietary”) to the individual andtherefore potentially useful in identifying one individual from another.Accordingly, the involuntary eye movements of the eyeball 100 are uniqueto an individual and can be used in user identification and userauthentication. The embodiments described herein disclose methods andapparatus to uniquely identify a user in response to eye motions.

Referring now to FIG. 3 , the small or miniature involuntary eyemovements of the eyeball 100 can be captured by a fixation process of aneye on a target image 301. The target image 301 becomes a retinal image302 on the retina 110 of the eyeball 100. The target image 301 can begenerated by a target 304 on a display device 310. The user stares at orfixates on the target 304 on the display device 310 for a period of time(e.g., ten to fifteen seconds) during the fixation process. Involuntarymovements are too small and subtle to be seen by direct observation withthe naked eye. A video camera 312 captures a sequence of images of themovement of the eyeball 100 during the fixation process.

To avoid capturing voluntary eye movement during a fixation process, onemay couple the camera 312 to the eyeball 100. However, this isimpractical for authentication purposes. Voluntary eye movement can besubstantially filtered out from the captured eye movement data togenerate involuntary eye movement data.

Furthermore, while miniature involuntary eye movement (also referred toherein as involuntary eye micro-motions) has typically been capturedduring a fixation process, it can also be captured with large scalevoluntary eye movement when the eyes are moving across a visual field,regardless of the focal point of interest. A video camera can be used tocapture a sequence of images of large scale eye movement that includesthe small scale or miniature involuntary eye movement. The miniatureinvoluntary eye movement can be extracted from the captured sequence ofimages-that include both involuntary and voluntary eye movement.

In FIG. 4A, zones of cones 400 are shown a magnified portion of thefovea centralis in the retina 110. FIG. 4A further illustrates a graphof involuntary eye movement over the zones of the cones in response to auser fixating on a target for about ten seconds. FIG. 4A represents aplurality of cones within a diameter of about five microns of the retina110.

Involuntary eye motions exist whether the subject or user fixates on astationary object/target or not. The small involuntary eye motionsinclude different types of eye movements, one of more of which can beused to identify a user.

FIG. 4A illustrates the different types of small involuntary eye motionsof interest, including saccades (also referred to as microsaccades)401A-401F, curve shaped drifts 402A-402E, and zig-zag shaped tremors(also referred to as micronystagmus) 403A-403E. FIG. 4B illustrates amagnified view of the saccades 401F, curve shaped drift 402E, and thezig-zag shaped tremor 403E. The zig-zag shaped tremors 403E are imposedon the curved shaped drifts 402E.

The drift eye movements 402A-402E are like a random walk without anyprecise aim when the eyes are not focusing on anything in particular.They are characterized by a small amplitude motions with changingdirections and by frequencies around 20-40 Hz. The tremor eye movementsare characterized by a very small amplitude (e.g., 0.2 to 2-3 degrees ofarc) and a higher frequency (e.g., 40-150 Hz, with 90 Hz being a typicalvalue). The saccadic eye movements are characterized by an amplitudebetween 15-20 degrees of arc, a high speed (between 200-500 degrees persec) and a relatively low frequency (e.g., from 0.1 Hz to between 1-5Hz).

The miniature-involuntary eye movements are a normal physiologicalfunction of the body to prevent fatigue of the rods and cones at thefocal point on the retinal surface within the eyeball. The image of atarget or object on the retina of the eye is constantly moving due tothe involuntary eye motions. The drift eye motion 402A-402E causes theimage to drift slowly outward away from the center of the fovea. Thedrift eye motion terminates at the start of the saccadic eye movement.The saccadic eye movement 401A-401F brings the image back towards thecenter of the fovea. The tremor eye motion 403A-403E, superimposed onthe drift eye motion 402A-402E, has an amplitude that crosses aplurality of cones to prevent exhaustion of a single cone when staringor fixating on the target.

These small involuntary eye movements are thought to prevent retinalfatigue. Retinal fatigue is the exhaustion of some key biochemical thatis essential to the capture of photons by the retinal cells. The smallinvoluntary eye movements are imperceptible to the naked eye. However,the involuntary eye movements can be captured andsensed/viewed/appreciated by retinal scanners and video cameras withsufficient speed to record eye movement, such as shown in FIG. 3 andFIG. 8B.

The involuntary eye motions/movements can be sensed by variousmeans/methods such as by using an infrared light emitting diode (LED),pupil scanning methods, even refractometry, provided that these methodsare modified to have the appropriate time/frequency and displacementsresolution to capture the involuntary eye motions/movements.

The involuntary eye movements can alternatively be sensed usingelectrooculography with sufficient sensitivity to generate anelectrooculogram (EOG) representative of the involuntary eye movement.The raw EOG signal originates in the dipole between the eye cornea andits retina.

Electrical signals also originate in the oculo-motor muscles when theycause the eye to generate eye movements. These electrical signalsoriginating in the oculo-motor muscles themselves can be sensed torepresent eye movement, similar to how to an electromyogram (EMG) fromthe eye.

Referring momentarily to FIG. 7 , e electrooculography involves sensingelectrical signals by measuring the potential difference (voltage)between the retina (grounded) and the cornea of the eye. Electrodes aretypically placed around the eye to measure up, down, left, and rightvoltages with respect to one or more ground electrodes. Differencesbetween left and right voltage measurements may be made to generate asignal indicating horizontal eye movement. Differences between up anddown voltage measurements may be made to generate a signal indicatingvertical eye movement. The signals can be amplified and filtered beforesignal processing occurs. The analog signals can be converted intodigital signals so that digital signal processing can analyze the EOGsignals for patterns in the involuntary eye movement of a user.

A pattern of the involuntary eye movement is employed as a method ofperforming a physiologic biometric identification of an individualbecause it is linked to the specific neuro-musculo-retinal anatomy ofsaid individual. Capturing the micro-motions of the involuntary eyemovements of the eyeball allows patterns to be extracted that are uniqueto the individual and can be used to authenticate the identity of anindividual. These extracted patterns are analyzed and features extractedthat provide a unique identifier of each individual. In accordance withone embodiment, a pupil is identified and tracked using a high speed,high resolution camera. The images are processed with a processorexecuting image processing software to extract the features from theinvoluntary eye movement that uniquely identify a user.

Involuntary eye movement can be used as a standalone method ofauthentication or in conjunction with retinal scanners that capture animage of the retinal anatomy of the eyeball. Involuntary eye movementcan be also used as a method of authentication in conjunction with irisscanners that capture an image of iris anatomy of an eyeball.Involuntary eye movement can be also used as a method of authenticationin conjunction with both retinal and/or iris scanners that captureimages of eye anatomy.

Using both involuntary eye movement and retinal and/or iris anatomyprovides for dual or triple authentication. Retinal and iris scanningtechnology does not include any neurologic component to thedetermination. Retinal scanning technology merely maps the anatomy ofthe retina to perform an authentication based on a two dimensional (2D)scan image. Involuntary eye movement adds a physiologic biometricparameter to the current eye scanning technology.

Prior art that analyzes eye movement can be distinguished from theembodiments disclosed herein.

United States (US) Patent Application Publication No. 2014/0331315,filed by Birk et al. on Dec. 23, 2011 (hereinafter Birk) discloses amethod for deriving a biometric parameter from eye movements for use inauthentication protocols; combining conventional authentication, such as“password or other information known to the user” as described in theAbstract, with eye movements based on a behavior repertoire consistingof actively moving the focus of the eye to “visually locate pieces ofinformation embedded in a display” as described in the Abstract. Thebehavior repertoire in Birk may be either the path of the eye movements,or the characterization of the user's eye movements as they sequentiallylocate a sequence of numbers/letters to match a password or code knownto the user.

Birk recognizes that there are several different types of eye movementsboth voluntary and involuntary. However, Birk describes using the onlyactive, voluntary eye movements as an input for the recognitiontechniques disclosed therein. Birk essentially uses a defined grid,across which the user eyes must wander in a specific sequence to berecognized. Birk also makes use of a repertoire of motions (habits)specific to a given user.

In contrast to Birk, the embodiments disclosed herein utilize only theinvoluntary movements of the eye that occur as part of the normal eyephysiology that are controlled by the brain stem and the cerebralcortex. The involuntary eye movements captured by the embodiments arenot based on what the user does, but based on what the user is.

U.S. Pat. No. 6,785,406 issued to Mikio Kamada on Aug. 31, 2004(hereinafter Kamada), describes an iris authentication apparatus forwhich they use some eye motions and iris contractions to ensure that thedata are collected from a live user and not from just an image or someother body part. The eye motions used in Kamada are cycloversions,motions linked to the vestibular control of the eyes that ensureeye-head coordination when one looks at an object while the head ismoving. Kamada also uses optokinetic nystagmus eye motion, which is alarge saccade that brings back the fovea of the retina to center when anobject drifts out of the visual field.

The eye motions described in Kamada are not micro-motions linked to theretinal fatigue and user-specific. The eye motions described in Kamadaare reflexive motions. Kamada does not use eye motions to identify auser but to ascertain whether or not the data captured is from a livecharacter. The eye motions described in Kamada are not user-specific sothey cannot be used to identify a user because they are reflexive arelike the knee jerk reflex, and present in every person, like a knee jerkreflex. Even though different strength levels may be recognized inKamada, they are not fine enough to discriminate between individuals toprovide identification.

U.S. Pat. No. 8,899,748 issued to Brandon Lousi Migdal on Dec. 2, 2014(hereinafter Migdal), describes a method to detect reflex nystagmus eyemotions linked to vestibular control (equilibrium) of an individual.However, vestibular nystagmus eye motions may be vertical or horizontaleye motions and originate as positional reflexes that are common.Moreover, vestibular nystagmus eye motions lack fine granularity ofinvoluntary eye micro-motions that are useful in discriminating betweenindividuals by the embodiments disclosed herein.

U.S. Pat. No. 9,195,890, issued to James R. Bergen on Nov. 24, 2015(hereinafter Bergen), discloses a biometric identification method basedon iris image comparisons. Bergen's comparison is based on extracted andunique anatomical features of the irises of eyes. The features of theirises are resolved at various depths of detail by Bergen. In order toalign and match an iris to a stored iris pattern, Bergen must correctfor motions and for tilts/positions, as well as correct for other imagedistortions. The eye motions referred to in Bergen are large scale andare undesirable because they represent a hindrance to acquiring qualitydata in Bergen's iris recognition system. This is contrary to theembodiments disclosed herein.

U.S. Pat. No. 7,336,806 issued to Schonberg et al. (hereinafterSchonberg); discloses iris-based recognition of a user for which theannulus circumference is key. Schonberg considers other features,including eye motion, to be noise that is to be eliminated.

U.S. Pat. No. 7,665,845 issued to Kiderman et al. on Feb. 23, 2010(hereinafter Kiderman) describes a video-oculographic (VOG) system thatis (VOG) based on light weight goggles. Kiderman's goggles were designedto make clinical measurements unbiased by the weight of the measuringinstrument. However, Kiderman does not disclose using involuntary eyemicro-motions that are of interest in the embodiments disclosed herein.Moreover, there is no obvious need for spatial resolution with regardsto the embodiments disclosed herein.

Eye Movement Detection

Electrooculography can directly generate signals of involuntary eyemovements. When using captured video images to track eye movements, aviewable feature of the eyeball in the video images is used. Patternrecognition may be used to detect the viewable feature in each image ofthe video images.

Referring now to FIGS. 6A, the viewable feature in the images of theeyeball may be the iris 114, the pupil 112, or the fovea 116. Edgedetection may be used to track involuntary eye movements from videoimages of the eyeball 100 captured while a user fixates on a target.Edge detection can be taken from the circumference 614 of the iris 114,the circumference 612 of the pupil 112, or the location of the fovea116.

As shown in FIG. 6B, the circumference 614 of the iris 114 is useful totrack involuntary eye movements because it is well defined andconsistent. The circumference 612 of the pupil 112 may alternatively beused to track involuntary eye movements. However, the size and locationof the pupil changes in response to light intensity.

As shown in FIG. 6C, the fovea 116 may alternatively be used to trackinvoluntary eye movements. However, tracking the fovea 116 typicallyrequires a light source shined through the lens 118 to illuminate theretina 110 of the eyeball 100.

Referring now to FIGS. 5A-5C, various combinations of the involuntaryeye movements can be used for user identification and authentication.

In FIG. 5A, all three types of involuntary eye movement, includingsaccade trajectories, drift, and tremors, are used to determine a uniqueidentifier of each user. In accordance with one embodiment, features areextracted from all three involuntary eye movements to uniquely identifyeach user. The system is consistent in extracting the same features overand over again for each user.

In FIG. 5B, two involuntary eye movements, such as saccade trajectoriesand drift, are used to determine a unique identifier of each user. Inaccordance with another embodiment, features are extracted from saccadetrajectories and drift to determine a uniquely identify each user. Thesystem is consistent in extracting the same features over and over againfor each user.

In FIG. 5C, a single involuntary eye movement, such as the tremorcomponent, is used to determine a unique identifier of each user. Inaccordance with yet another embodiment, features are extracted from thetremor component of the involuntary eye movements to uniquely identifyeach user. The system is consistent in extracting the same features overand over again for each user.

Referring now to FIG. 7 an exemplary electrooculography system 700 isshown to directly generate signals of the involuntary eye movements of auser 799. A plurality of electrodes 701A-701E are applied near the eyesaround a users head/face to capture voltages around each eye during eyemovement. The electrodes may be part of a hood or a face receivingdevice to couple the electrodes to the surface of the user's head/face.Electrodes 701A-701B capture the up and down or vertical motion of theeyeball 100. Electrodes 701C-701D capture the left and right orhorizontal motion of the eyeball 100. One or more electrodes 701Eprovide a ground or zero voltage reference for each electrode 701-701D.The electrodes 701A-701D measure up, down, left, and right voltages withrespect to the one or more ground electrodes 701E as involuntary eyemovement occurs during the fixation process.

The up and down voltages of electrodes 701A-701B with or withoutfiltering are coupled into the negative and positive inputs of adifference amplifier 704A to amplify and compute the difference betweenthe up and down voltages. This forms a vertical eye movement signal. Thevertical eye movement signal may be coupled into analog filters 706A toremove noise and other unwanted signals to additionally emphasize thevertical eye movement signal. The filtered vertical eye movement signal,an analog signal, is coupled into a analog to digital converter 708A toconvert the analog form into a digital form of signal. The digitalfiltered vertical eye movement signal is coupled into a first paralleldigital input of a digital signal processor 710. In an alternateembodiment, the signal processor can be manufactured to support mixedanalog and digital signals. Accordingly, the signal processor 710 caninclude be used

Similarly, left and right voltages of electrodes 701C-701D with orwithout filtering are coupled into the negative and positive inputs of adifference amplifier 704B to amplify and compute the difference betweenthe left and right voltages. This forms a horizontal eye movementsignal. The horizontal eye movement signal may be coupled into analogfilters 706B to remove noise and other unwanted signals to additionallyemphasize the horizontal eye movement signal. The filtered horizontaleye movement signal, an analog signal, is coupled into a analog todigital converter 708B to convert the analog form into a digital form ofsignal. The digital filtered horizontal eye movement signal is coupledinto a second parallel digital input of the digital signal processor710.

The digital signal processor 710 performs digital signal processingusing both the digital filtered horizontal eye movement signal and thedigital filtered vertical eye movement signal to analyze, extractfeatures, and generate the unique identifying pattern from theinvoluntary eye movement of the user. In this case, the uniqueidentifying pattern of involuntary eye movement is directly capturedfrom the user's head/face without using a video camera or scanner andanalyzing images.

Referring now to FIG. 8A-8B, an electronic device 800 with a videocamera is used to capture images of eye movement of the user 899 duringthe fixation process.

In FIG. 8B, the electronic device 800 includes a display device 807 anda video camera 812 coupled to a processor, microcomputer, ormicroprocessor (up) 801. The electronic device 800 further includes amemory 802 to store program instructions and user associated data. Theelectronic device 800 may further one or more (radio frequencytransmitters/receivers) radios 809 and one or more wired connectors822,824 (e.g., USB port 822, and/or network interface port 824) toprovide communication between the electronic device 800 and otherelectronic devices by wireless or wired means. The electronic device 800further includes a touch screen display device 807 to provide adisplayable user interface UI to the user using the electronic device800. Software controls can be displayed on the touch screen displaydevice 807 so the user can control the electronic device. The electronicdevice 800 can optionally include one or more hardware buttons 804 tofurther allow the user to control the electronic device.

In support of capturing eye movement, the processor 801 generates afixation target 850 that is displayed by the display device 807. A useris asked to fixate on the fixation target 850 while the video camera812, under control of the processor 801, captures a temporal sequence ofimages, a video, of the users eyes. The video, from frame to frame,captures the involuntary eye movement of one or both eyeballs of theuser. 3D accelerometer data is captured with the video to removephysical movements of the camera 812 and the electronic device 800 fromdetermining eye movement.

The processor 801 includes or may be adapted to provide signal processorfunctionality. In any case, the processor executes instructions ofpattern recognition software and signal processing software to analyzethe video capturing the involuntary eye movement of one or both eyeballsof the user. The pattern recognition software may be used to identifythe eyeballs in the video and then the irises and pupils in the video.

The captured video is analyzed to detect if a user blinks during thetemporal sequence of images and whether or not a sufficient sequence iscaptured detect involuntary eye movements. If not, the user is asked bythe user interface to repeat the fixation process.

If a sufficient sequence of images is captured, further analysis isperformed to determine the movement of the eyeball from image to image.A reference point on the eyeball, such as the iris or pupil, is used todetermine the involuntary movement in the captured video of the fixationor staring process. The 3D accelerometer data is used to exclude thephysical movement of camera 812 captured in the video from the raweyeball movement data to form true eyeball movement data.

The true eyeball movement data is further analyzed to extract thedesired type of involuntary eye movement data that is to be used inauthenticating and/or uniquely identifying the user.

A user is initialized to the electronic device 800 to store an initialdata set of initial captured involuntary eye movement data. From theinitial data set, features linked to the time-series of the involuntaryeye motions are extracted and classified to be associated with the user.

Subsequently features extracted from captured data sets of newlycaptured involuntary eye movement data are compared against theassociated stored features extracted from the initial capturedinvoluntary movement data to identify a user. A match percentage can becalculated to determine if the user is authorized to use the electronicdevice.

The newly captured involuntary eye movement data is compared against theinitial captured involuntary movement data to determine match results.If the match results are within a match percentage, the user isidentified and authorized to use the device. If the match results areoutside the match percentage, the user is unidentified and notauthorized to use the device.

Involuntary eye motions (e.g., tremors) can occur at about a maximumfrequency of 150 Hertz (Hz) or cycles per second. Cones of an eye rangein diameter from 0.5 to 4 micro-meters (microns). To capture the desiredinvoluntary eye motions, appropriate recording devices in systems, suchas the video camera 812 coupled to the processor 801 in the electronicdevice 800, operate at a minimum frame rate (frames per second) of atleast a range of 300 Hz-450 Hz and optimally at a frame rate of 1000 Hzor more. Cameras and processors in pre-existing electronic devices maybe re-programmed by a software application or driver to run faster thanthe typical setting. Normal (large scale saccade) covers 300 degrees persecond and can re-align the eyes within a third of a second. Whereasinvoluntary micro-saccades cover a few degrees per second down to 0.2degrees in amplitude. Accordingly, the spatial resolution of recordingdevices that can resolve down to within the range of 0.1-0.2 degrees canbe optimal for capturing the desired involuntary eye motions.

While one type of electronic device 800 is shown in FIGS. 8A-8B forcapturing the desired involuntary eye motions, other types of electronicdevices can be used to capture the desired involuntary eye motions.FIGS. 9A-11C illustrate other means and electronic devices to capturethe desired involuntary eye motions.

Referring now to FIGS. 9A-9B, electronic glasses 900 are shown that maybe used to capture images of eye movement of a user. From the capturedimages of eye movement, the small scale involuntary eye motions can beextracted. The electronic glasses 900 may be temporarily worn by a userin order to identify, authenticate, and authorize a user to a system orapparatus.

The electronic glasses 900 include an eye glass frame 902 to which aleft lens 904L and a right lens 904R are mounted in a pair of eye wires.In accordance with one embodiment, the eye glass frame 902 includes abridge 907, a left temple 908L, a right temple 908R, nose pads, nose padarms, and the pair of eye wires. They electronic glasses 900 mayalternatively be clip on glasses worn over prescription glasses. Thelenses 904L-904R may be prescription lenses or not.

A small target 906 may be formed in an upper right corner of the leftlens 904L to direct a user's eyes towards a video camera. Alternatively,the target 906 could be formed in an upper left corner of the right lens904R to direct the user's eyes towards the video camera. The target 906can be formed on either lens by printing a target image onto the surfaceof the lens, by inscribing the target image into the lens, by shining atarget light onto the surface of the lens; or by other known means ofapplying an image onto a clear surface.

Referring momentarily to FIG. 9C, the target 906 is a shortdepth-of-field target with a center opening or hole 907. With the target906 on the lens 904L or 904R, it is too close for the user to focus on.However, the user can focus through the center opening 907 of the target906. The target 906 with its center opening 907 acts like an opticaltether so that the pupil is located in line with the video camera tobetter capture eye movement.

Referring now to FIG. 9B, the electronic glasses 900 further includesthe video camera 910, a processor 912, a memory device 914, a radiotransmitter/receiver (transceiver) 916, and a power supply (e.g.,battery) 920 mounted to the eye glass frame 902. The electronic glasses900 may further include an optional light emitting diode (LED) 918mounted to the frame 902 or a nose pad arm 922.

The video camera 910 is angled slightly towards the lens with thetarget, such as the left lens 904L with the target 906, so that it ismore in line with the eye when focusing on the target. The video camera910 and processor 912 operate together at a frame rate in the range of a300 to 1000 frames per second to capture involuntary eye motions at amaximum frequency of 150 Hz.

The radio 916 may be used by the processor to communicate with acomputer or server to authenticate the user to the computer or server.The memory 914 may be used to store initialization data, including theinitial involuntary eye motion features that are extracted from thecaptured eye motions.

The electronic eyeglasses 900 may be temporarily worn by the user toauthenticate the user to a system. In the alternative, electronicgoggles may be used.

Referring now to FIGS. 10A-10B, electronic virtual reality headset orgoggles 1000 including a video camera that may be used to capture eyemotion of an eye of a user. The electronic virtual reality headsetincludes a frame 1002 and a head strap 1003 to retain the headsetaffixed to the users head. The frame 1002 includes top, bottom, left,and right flexible curtains or blinders 1004 configured to receive theface around the eyes of the user to provide a hooded area 1099 to keepoutside light from entering. The bottom curtain or blinder 1004Bincludes a nose opening 1005 to receive the nose of a user.

In FIG. 10B, the headset 1000 further includes the video camera 1010, aleft display device 1012L, and a right display device 1012R coupled tothe frame 1002. The headset 1000 further includes a processor 1011 and amemory 1014 coupled together. The video camera 1010 and the left andright display devices 1012L,1012R are coupled to the processor 1011. Theleft display device 1012L and the right display device 1012R can providea stereo three dimensional image to the user at varying perceiveddepths. The video camera 1010 may be coupled to the frame 1002 insidethe hooded area 1099 at different locations to capture eye motion. Thevideo camera 1010 may be located on the right as shown to capture eyemotion to avoid interfering with the video images displayed by the leftand right display devices 1012L,1012R. In an alternate embodiment, apair of video cameras 1010 may be located on opposite sides to captureeye motions of both left and right eyes so that the involuntary eyemicro-motions from one or both eyes are used to authenticate a user.

A stereo three dimensional target comprising a left target 1006L and aright target 1006R may be generated by the processor 1011 and displayedon the left display device 1012L and the right display device 1012R,respectively. The left target 1006L and the right target 1006R can causethe target to appear far away to focus the eyes at a distant andsomewhat fixate the eyes to better capture involuntary eye movement withthe video camera 1010.

The processor 1011 may be wired by a cable 1050 and plug 1052 to anothersystem. Alternatively, a radio transmitter/receiver (transceiver) 1016may be coupled to the processor 1011 so that the processor andheadset/goggles can wirelessly be coupled to another system to use theauthentication capability of the headset/goggles 1000.

Referring now to FIG. 11A, eye motion can be captured in another mannerby using a contact lens 1100 mounted to one or both eyes 100 of a userover the lens 118. The contract lens 1100 includes one or more emitters1102 coupled to (e.g., embedded in or printed on) the lens material1104. The emitter 1102 may be an active device, such as a light emittingdiode or a radio beacon with associated driving circuits (e.g., radiotransmitter/receiver, diode driver); or a passive device, such as areflector, retro-reflector, or mirror.

In the case of an active emitter device, one or more sensors are used toreceive the emitted light or radio signal to determine position of theeye from one time point to the next to directly capture eye motion overa period of time. Power may be wirelessly coupled from a base antennaaround the eye into an antenna coupled to the active emitter device inthe contact lens. Radio signals may also be coupled between the baseantenna and the antenna coupled to the active integrated circuit device.A three dimensional motion sensor may be included as part of theintegrated circuit to capture eye motion including the involuntary eyemicro-motions of interest.

In the case of the passive emitter device, light of a light source isdirected to the passive emitter device to activate it into reflectinglight back to one or more photo diode sensors around the eye. Acombination of active emitter devices and passive emitter devices may beused in the same contact lens to capture eye motion by either or bothmeans.

FIGS. 11B-11D illustrate one emitter 1102A, two emitters 1102B-1102C,and four emitters 1102D-1102G embedded in contact lenses 1100A-1100C,respectively.

In FIG. 11B, an active emitter 1102A is depicted coupled to two or moreantenna lines 1105A-1105B around a segment of the circumference edge ofthe contract lens 1100A. The active emitter 1102A includes an integratedcircuit 1106 with a processor/controller and other circuitry externallycoupled to it or internally integrated on the integrated circuit.Alternatively, the emitter 1102A may be a passive emitter.

In FIG. 11C, a pair of passive emitters 1102B-1102C are depicted arounda segment of the circumference edge of the contract lens 1100B.Alternatively, the emitters 1102B-1102C may be active emitters with twoor more antenna feed 1105A-1105B in a segment near the circumferenceedge of the contract lens 1100B.

In FIG. 11D, a pair of active emitters 1102D-1102E and a pair of passiveemitters 1102F-1102G are shown near the circumference edge of thecontract lens 1100C. Alternatively, all emitters 1102D-1102G may beactive emitters or passive emitters; just one may be passive with allothers active; or just one may be active with all others passive.

Referring now to FIG. 11E, further details of an instance of an activeemitter 1102 are shown. The active emitter 1102A includes an integratedcircuit 1106 with a processor/controller 1110 and other circuitryexternally coupled to it or internally integrated on the integratedcircuit coupled to the processor/controller 1110.

The integrated circuit (IC) 1106 can receive power over the antenna1105A-1105B into a power conversion circuit 1112 coupled to theprocessor 1110. Radio frequency energy from an oscillating radiofrequency (RF) signal is inductively coupled into the two or moreantenna feeds 1105A-1105B by a nearby base antenna. The power conversioncircuit 1112 can rectify and regulate the AC RF signals into a DC powersource for other circuits within the IC 1106 as well as those coupled toit.

With a light emitting diode (LED) 1108 coupled to the integrated circuit1106, a diode driver 1118 therein is coupled to and between theprocessor 1110 and the light emitting diode (LED) 1108. With power beinggenerated by the power conversion circuit, the processor 1110 cangenerate a signal to activate the diode driver 1118. With power, theprocessor can activate the diode driver 11118 to drive and provide powerto the light emitting diode 1108 to emit light out away from the eye ofthe user.

Alternatively or in addition to, the integrated circuit 1106 may have a3D motion sensor 1117 coupled to the processor that directly senses eyemotion. With power, the processor coupled to a radiotransmitter/receiver (transceiver) 1109 can transmit and receive radiosignals through the radio transceiver 1119 over the antenna lines1105A-1105B. The base unit with its own corresponding radio transceivercan collect and further process the eye motion data.

Referring now to FIG. 11F, for an active emitter, a base unit 1199 isshown including a frame 1121, a lens 1122, and poles of a base antenna1124A-1124B wrapped around the lens 1122. Wires 1114 from the baseantenna 1124A-1124B are coupled to a base radio receiver/transmitter(transceiver) (not shown), and then to a base processor (not shown) toprocess the captured eye motion signals.

With the base antenna 1124A-1124B of the base near the antenna lines1105A-1105B of the contact lens, they can be inductively coupledtogether to transfer radio frequency power/energy as well as radiofrequency signals between the base unit 1199 and the contact lens 1100C.When powered up, the eye motion captured by the motion sensor 1117 canbe the communicated from the contact lens 1100C to the base unit 1199 bythe radio transceivers in each.

For a passive emitter, the base unit 1199 (additionally oralternatively) includes one or more photo diode sensors 1134A-1134Bcoupled to the lens 1112 near its edges. The base unit 1199 may furtherinclude a light source 1132, such as a display device, to shine lighttowards one or more passive emitters 1102F,1102G. The light reflectingoff the one or more passive emitters 1102F,1102G is captured by the oneor more photo diode sensors 1134A-1134B to determine position andmovement of an eye over time. Wires 1114 from the photo diode sensors1134A-1134B are coupled to a base processor (not shown) to process thecaptured eye motion signals.

The base unit 1199 may be in the form of glasses (spectacles), VRgoggles, a stand alone eye scanner, or a wall mounted eye scanner.

While an electronic device may be worn by a user adjacent a user's eyeor eyes, such as in the case of glasses, goggles, and contact lenses;the electronic device to capture eye motions may be supported by astructure or a system with the user placing his/her eye or eyes near avideo camera, sensors, or antenna to capture eye motions that includethe involuntary eye micro-motions of interest.

FIG. 12A illustrate an eye motion capture device 1201 affixed to abuilding structure 1200 to control access to one or more doors 1202 inresponse to the involuntary eye micro-motions of a user. The eye motioncapture device 1201 is coupled to an access system 1203 to controlaccess to the one or more doors 1202 of the structure 1200. The accesssystem 1203 can control unlocking one or more doors 1202 in response toproper involuntary eye micro-motions of the eye of an authorized user.

FIG. 12B illustrates a magnified view of the eye motion capture device1201 that receives the area of the face of a user around the left orright eye. As discussed previously with reference to FIG. 10B, the eyemotion capture device 1201 may include some similar structure andfunction of a processor 1011, as well as a video camera 1010, a displaydevice 1012R and a memory 1014 coupled to the processor. The processor1011 may be wired by a cable and a plug to the access system 1203.Alternatively, the processor 1011 may be wirelessly coupled to theaccess system 1203 by a radio 1016 coupled to the processor 1101.

FIG. 13A illustrates a stand alone eye scanner 1301 coupled to a system1302 by wire with a cable or wirelessly with radiostransmitter/receivers in each. The system 1302 may be coupled to aserver 1306. The server may be remote and accessed over a wide areanetwork 1304, such as the internet. The stand alone eye scanner 1301 canbe used to non-invasively authenticate the user to the system 1302, andthe server 1306, in response to the involuntary eye micro-motions of oneor more eyes.

FIG. 13B illustrates a magnified view of the stand alone eye scanner1301 that receives the eye area of the face of a user. As discussedpreviously with reference to FIG. 10B, the eye motion capture device1301 similarly includes the structure and function of a processor 1011,as well as one or more video cameras 1010, a left display device 1012L,a right display device 1012R, and a memory 1014 coupled to theprocessor. The processor 1011 may be wired by a cable 1050 and a plug1052 to the system 1302. Alternatively, a radio transmitter/receiver(transceiver) 1016 may be coupled to the processor 1011 so that theprocessor and headset/goggles can wirelessly be coupled to the system1302. Left and/or right targets 1006L,1006R can be similarly generatedon the left and/or right display devices 1012L,1012R so that the standalone eye scanner 1301 can scan one or both eyes to non-invasivelyauthenticate the user in response to the involuntary eye micro-motionsof one or both eyes of the user.

In each of the electronic devices, the processor cooperates with anotherdevice to capture a representation of user eye movement from which thedesired involuntary eye micro-motions can be extracted. The processormay further perform signal processing on the extracted involuntary eyemicro-motions to determine identifying eye micro-motion features fromthe extracted involuntary eye micro-motions that are extracted andselected repeatedly by the same system, such as described in U.S. patentapplication Ser. No. 15/013,875; filed by Martin Zizi et al. on Feb. 2,2016, incorporated herein by reference.

Identifying eye micro-motion features can be used with various systemsthat utilize user authentication. For example, the identifying eyemicro-motion features can be classified in by a match percentage andused to authenticate a user with an authentication controller such asshown and described in U.S. patent application Ser. No. 15/013,875;filed by Martin Zizi et al. on Feb. 2, 2016, incorporated herein byreference. The identifying eye micro-motion features can be used toprovide keyless access to homes, buildings, and vehicles such as shownand described in U.S. patent application Ser. No. 15/013,810; filed byMartin Zizi et al. on Feb. 2, 2016, incorporated herein by reference.The identifying eye micro-motion features can be used to encrypt/decryptdata such as shown and described in U.S. patent application Ser. No.15/013,792; filed by Martin Zizi et al. on Feb. 2, 2016, incorporatedherein by reference. The identifying eye micro-motion features can beused to secure access to privacy data, such as medical records shown anddescribed in U.S. patent application Ser. No. 15/013,764; filed byMartin Zizi et al. on Feb. 2, 2016, incorporated herein by reference.

CONCLUSION

The embodiments of the invention are thus described. When implemented insoftware, the elements of the embodiments of the invention areessentially the code segments or instructions to perform the necessarytasks. The program or code segments/instructions can be stored in aprocessor readable medium for execution by a processor, such asprocessor 801. The processor readable medium may include any medium thatcan store information, such as memory 802. Examples of the processorreadable medium include an electronic circuit, a semiconductor memorydevice, a read only memory (ROM), a flash memory, an erasableprogrammable read only memory (EPROM), a floppy diskette, a CD-ROM, anoptical disk, or a hard disk. The program and code segments/instructionsmay be downloaded via computer networks such as the Internet, Intranet,etc.

While this specification includes many specifics, these should not beconstrued as limitations on the scope of the disclosure or of what maybe claimed, but rather as descriptions of features specific toparticular implementations of the disclosure. Certain features that aredescribed in this specification in the context of separateimplementations may also be implemented in combination in a singleimplementation. Conversely, various features that are described in thecontext of a single implementation may also be implemented in multipleimplementations, separately or in sub-combination. Moreover, althoughfeatures may be described above as acting in certain combinations andeven initially claimed as such, one or more features from a claimedcombination may in some cases be excised from the combination, and theclaimed combination may be directed to a sub-combination or variationsof a sub-combination. Accordingly, the claimed invention is limited onlyby patented claims that follow below.

1-19. (canceled) 20-61. (canceled)
 62. A method for online services, themethod comprising displaying a webpage of an online service; receiving auser name associated with a user; receiving a token from the user, thereceived token associated with a neurological eye print that uniquelyidentifies the user; and in response to the received token, allowing theuser to access the online service.
 63. The method of claim 62, whereinthe online service is a health care service to authenticate the userprior to access of electronic medical records associated with the user;and the online service allows the authenticated user to send and receiveelectronic medical records, portions of electronic medical records, ormedical data associated with the authenticated user.
 64. The method ofclaim 63, wherein the electronic medical records are universal healthrecords having a standardized format.
 65. The method of claim 62,wherein the online service is a government online service; and the tokenassociated with the neuro-mechanical fingerprint is date and timestamped to provide a current positive identification of the user. 66.The method of claim 65, wherein the online service is an online votingservice.
 67. The method of claim 65, wherein the online service is anonline testing service.
 68. The method of claim 62, further comprising:looking up user calibration parameters from a storage device in responseto the name of the user; generating a number in response to the tokenand the user calibration parameters; determining a match percentage inresponse to the generated number; in response to the match percentage,determining if the user is an authorized user by comparing the matchpercentage to an authorized user percentage; and in response to thematch percentage being greater than or equal to the authorized userpercentage, allowing access of the online services to the user.
 69. Themethod of claim 68, wherein the generated number is further generated inresponse to a date and time stamp within an expected range of date andtime stamps.
 70. The method of claim 68, wherein the token is encryptedwith an encryption code and the generating of the number furthercomprises prior to generating the number, decrypting the token inresponse to the encryption code.
 71. The method of claim 69, wherein thegenerated number is further generated in response to a date and timestamp within an expected range of date and time stamps.
 72. The methodof claim 70, wherein the decryption of the token is further in responseto a date and time stamp of the token within an expected range of dateand time stamps.
 73. The method of claim 68, further comprising: priorto receiving the token, storing the user calibration parameters in thestorage device.
 74. A method of securing data, the method comprising:capturing eye motion of an eye of a first authorized user to generate afirst multi-dimensional eye motion signal; extracting involuntary eyemotion from the first multidimensional eye motion signal; generating afirst neurological eye print in response the involuntary eye motion andfirst user calibration parameters; and encrypting data with anencryption algorithm using the first neurological eye print as anencryption key.
 75. The method of claim 74, further comprising accessingthe encrypted data that is encrypted with the first neurological eyeprint as the encryption key including capturing eye motion of an eye ofa first authorized user to generate a second multi-dimensional eyemotion signal; extracting involuntary eye motion from the secondmultidimensional eye motion signal; regenerating the first neurologicaleye print in response to the second the multi-dimensional eye motionsignal; determining a match percentage in response to the firstregenerated neurological eye print and the first user calibrationparameters; and in response to the match percentage being greater thanor equal to an access match level, decrypting data with the encryptionalgorithm using the first regenerated neurological eye printrepresenting the first encryption key.
 76. The method of claim 74,further comprising: storing the first neurological eye print in asecured storage device associated with the first neurological eye printand one or more secondary neurological eye prints, wherein the storedneurological eye print is accessible from the secured storage device inresponse to the first neurological eye print regenerated from the firstauthorized user, and wherein the stored neurological eye print isfurther accessible from the secured storage device in response to theone or more secondary neurological eye prints generated from one or moresecondary authorized users differing from the first authorized user. 77.The method of claim 76, wherein the first authorized user invites theone or more secondary authorized users to have access to the storedneurological eye print in response to verification of the one or moresecondary authorized users with their respective one or more secondaryneurological eye prints.
 78. The method of claim 76, further comprising:accessing the encrypted data that is encrypted with the firstneurological eye print as the encryption key including capturing eyemotion of an eye of a one or more secondary authorized users to generatea second multi-dimensional eye motion signal; extracting involuntary eyemotion from the second multidimensional eye motion signal; in responseto the second multi-dimensional eye motion signal, regenerating a secondneurological eye print of the one of the one or more secondaryauthorized users; determining a live match percentage in response to thesecond regenerated neurological eye print and the second usercalibration parameters associated with the one of the one or moresecondary authorized users; and in response to the live match percentagebeing greater than or equal to a live access match level, verifying theone of the one or more secondary authorized users to provide access ofthe stored neurological eye print in the secured storage device.
 79. Themethod of claim 78, wherein the accessing of the encrypted data that isencrypted with the first neurological eye print as the encryption keyfurther includes in response to verification of the one of the one ormore secondary authorized users, accessing the stored neurological eyeprint from the secured storage device; determining a stored matchpercentage in response to the stored neurological eye print and thefirst user calibration parameters; and in response to the stored matchpercentage being greater than or equal to a stored access match level,decrypting data with the encryption algorithm using the storedneurological eye print representing the encryption key.
 80. The methodof claim 79, wherein the stored access match level is less than or equalto the live access match level.
 81. The method of claim 76, wherein themulti-dimensional motion of the body part is at least two dimensionalmotion and the multi-dimensional signal is at least a two dimensionalsignal.
 82. The method of claim 75, wherein the encrypted data is storedon a remote server; and the first user is remotely verified over acomputer network to decrypt the encrypted data.
 83. The method of claim79, wherein the encrypted data is stored on a remote server; and the oneof the one or more secondary authorized users is remotely verified overa computer network to access the stored neurological eye print in thesecured storage device and decrypt the encrypted data.
 84. The method ofclaim 76, wherein the secured storage device is a portable securedstorage device. 85-104. (canceled)